Computer systems and networks require backup and archival of data as part of their regular maintenance and security. To address a need for data backups in computer systems and networks using fewer server and local area network (“LAN”) resources, the concept of serverless backup was devised. Serverless backup is also commonly referred to as “third party copy.” This technology is widely known within the computer storage industry, and is detailed in specification T10/99-143r1 (“T10”) and, more recently, in SCSI Primary Commands-3 (“SPC-3”), which specifications are incorporated herein by reference.
In a traditional LAN free backup, computer data is backed up using a SAN. While this frees the associated LAN from the data traffic occasioned by the backup, it still burdens one of the servers on the SAN with copying data from various storage devices to various backup storage devices. An example of a traditional LAN free backup is illustrated in FIG. 1. FIG. 1 depicts three servers (11, 12, 13) connected to a series of physical storage devices (15, 16, 17, 19, 20) through a generic SAN interconnect topology (10) and a storage router (18). The storage devices in this example are SCSI tape drives (16, 17) and disk arrays (15), and Fibre Channel tape drives (20) and disk arrays (19). An associated LAN is also depicted (14). FIG. 1 also illustrates the flow of commands (21, 23, 25) and data (22, 24, 25) to and from such physical storage devices.
In serverless backup, a specialized copy manager, or data mover, handles all of the data movement, or transfers, associated with the data backup. Examples of such data transfers include block to stream, stream to block, block to block, inline to stream, etc., all of which are described in the T10 and/or SPC-3 specifications. Serverless backup removes servers from the data path during the backup process, thus enabling those servers to be utilized for tasks other than backing up data. In a serverless backup system, a server, commonly referred to as a “host” in the SAN context, creates and transfers a command, i.e. an EXTENDED COPY command, to a copy manager residing in the serverless backup system. The copy manager handles all data transfers associated with the backup, and reports status back to the host that issued the command. An example of a serverless backup system is illustrated in FIG. 2. Such systems are disclosed in the prior art.
In FIG. 2, the serverless backup system (40) is depicted as part of a SAN (30). As in FIG. 2, serverless backup systems are typically associated with one or more hosts (42, 43, 44). Hosts associated with serverless backup systems may run data backup software sold by any one, or a combination of, numerous vendors to generate serverless backup commands. Such hosts include a wide array of hardware and operating system platforms.
Serverless backup systems typically include, without limitation, one or more copy managers (50) that perform backups according to serverless backup commands, and one or more physical storage devices (46, 47, 48, 51, 52) that data is transferred to and/or from during the serverless backup process. The physical storage devices depicted in FIG. 2 are of the same variety as those depicted in FIG. 1, as described above. Likewise, FIG. 2 depicts a SAN interconnection topology (41) and the flow of commands (53, 55, 59) and data (54, 56, 59) to and from such physical storage devices.
The copy manager (50) in the example of FIG. 2 resides within a storage router (49) for illustrative purposes. In practice, a copy manager may reside in a broad spectrum of devices. These devices include, but are not limited to, storage routers, bridges, switches, hubs, and individual physical storage devices (e.g. tape drives, hard drives, disk arrays, CD drives, etc.). The main criteria for defining a copy manager is any device, with the exception of hosts, capable of processing a serverless backup command.
Storage devices associated with serverless backup commands may be divided into two broad categories: physical storage devices and virtual storage devices.
Physical storage devices are actual, hardware-based devices that are used to store data. Examples are the Fibre Channel (52) and SCSI (47,48) tape drives and hard disk arrays (46, 51) depicted in FIG. 2. Further examples of physical storage devices include individual disk drives, tape drives, CD drives, DVD drives, arrays of such devices, and others. Storage protocols, and related storage protocol commands, used in interfacing the physical storage devices vary based upon application. Some examples are SCSI, Fibre Channel, iSCSI, ATA, etc.
Unlike physical storage devices, virtual storage devices do not exist in any clearly visible form, but are instead conceptual components built into the copy manager. For serverless backup commands, some examples of virtual devices include the PAD, INLINE, EMBEDDED, and DISCARD devices or associated actions. These virtual storage devices enable special case processing that may be called for in connection with a given serverless backup command.
Both physical storage devices and virtual storage devices may act as the sources of data to be transferred in the context of a serverless backup command. In such a capacity, such storage devices are referred to herein as “source storage objects.” In the example depicted in FIG. 2, both the SCSI and Fibre Channel disk arrays may act as source storage objects. Similarly, both physical storage devices and virtual storage devices may act as the destinations to which data is transferred in the context of a serverless backup command. In such a capacity, they are referred to herein as “destination storage objects.” In the example of FIG. 2, the Fibre Channel tape drive (52) acts as a destination storage object.
A serverless backup command (57) consists of an identifier for a command along with associated parameters, or command flags. Such a command also contains a list of target descriptors which describe the storage devices (targets) to be acted upon in processing the command. A serverless backup, or EXTENDED COPY, command further contains a list of segment descriptors that describe the data movements to occur with relation to those targets. Segment descriptors can be viewed generally as instructions as to how to perform the desired backup with the relevant storage devices. Segment descriptors are referred to generically herein as “instructions.” The components of such instructions include, among other things, the location of the data to be transferred, the storage devices involved, the size of the data to be transferred, and others, all of which are described in the T10 and SPC-3 specifications incorporated herein. In addition to target and segment descriptors, a command may contain inline data to be transferred as part of the backup process.
The basic structure of a serverless backup command may be depicted as follows:
COMMAND IDENTIFIERCOMMAND FLAGSTARGET DESCRIPTOR LIST LENGTHSEGMENT DESCRIPTOR LIST LENGTH1ST TARGET DESCRIPTOR* * *NTH TARGET DESCRIPTOR1ST SEGMENT DESCRIPTOR* * *NTH SEGMENT DESCRIPTORINLINE DATA
The detailed format of such serverless backup commands varies based upon specification, and are included in specifications T10 and SPC-3.
In existing serverless backup command processing, a copy manager moves or transfers data by processing each segment descriptor (describing the data transfer to be performed) in the order in which it is received, i.e. in a linear fashion. In doing so, the copy manager reads data from a source storage device and writes data to a destination storage device for an individual segment. A flow chart illustrating such an existing process is depicted in FIG. 3. Using this method to process the command (60, 61, 62, 63, 64, 65, 66), copy managers allocate internal resources as required for the particular instance of data movement (segment).
While such traditional command processing may reliably back up data, it is performed in a slow and cumbersome manner. Delays may result in not only the execution of a command, but also in the retrieval of data, because traditional methods do not organize and segment the segment descriptors or the corresponding data in a preferred manner. No effort is made in such existing systems to increase the efficiency of copy managers or storage devices performing the backup, i.e. transferring or moving data. A further limitation of traditional processing is that individual segment descriptors, and the corresponding data transfers, are limited by the size of physical memory within a copy manager.
Therefore, a need exists for a method to address these shortcomings and improve the efficiency and ensure the integrity of data transfers in serverless backup systems. The methods of this invention were devised to address this need.